User-friendly Tail Bounds for Matrix Martingales

This report presents probability inequalities for sums of adapted sequences of random, self-adjoint matrices. The results frame simple, easily verifiable hypotheses on the summands, and they yield strong conclusions about the large-deviation behavior of the maximum eigenvalue of the sum. The methods also specialize to sums of independent random matrices. 1. Main Results This technical report is a companion to two other works, the papers “User-friendly tail bounds for sums of random matrices” [Tro10c] and “Freedman’s inequality for matrix martingales” [Tro10a]. Since this report is intended as a supplement, we have removed most of the background discussion, citations to related work, and auxiliary commentary that places the research in a wider context. We recommend that the reader peruse the original papers before studying this report. The paper [Tro10a] describes a martingale technique that leads to an extension of Freedman’s inequality in the matrix setting, which is similar to the result [Oli10a, Thm. 1.2]. The purpose of this work is to show how the arguments from [Tro10a] allow us to establish the matrix probability inequalities for sums of independent random matrices that appear in [Tro10c]. The discussion here also contains some new probability inequalities for sums of adapted sequences of random matrices; we have removed these results from the other two papers because they are somewhat specialized. 1.1. Roadmap. The rest of the report is organized as follows. The balance of §1 provides an overview of the main results for sums of independent random matrices. Section 2 contains the main technical ingredients for the proof. Sections 3–5 complete the proofs of the matrix probability inequalities for adapted sequences. Appendix A provides an overview of the background material that we require. 1.2. Rademacher and Gaussian Series. Let ‖·‖ denote the usual norm for operators on a Hilbert space, which returns the largest singular value of its argument, and let λmax denote the algebraically largest eigenvalue of a self-adjoint matrix. The extreme eigenvalues of a Rademacher series with self-adjoint matrix coefficients exhibit normal concentration. Theorem 1.1 (Matrix Rademacher and Gaussian Series). Consider a finite sequence {Ak} of fixed self-adjoint matrices with dimension d, and let {εk} be a finite sequence of independent Rademacher variables. Compute the norm of the sum of squared coefficient matrices: σ := ∥∥ ∑ k Ak ∥∥ . (1.1) Date: 25 April 2010. Revised on 15 June 2010, 10 August 2010, 14 November 2010, and 16 January 2011.